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Abstract 


Playful learning is an approach that uses play to support discovery learning and 
problem-solving. In an online learning environment that aims to teach STEM education, 
incorporating playful learning is a challenge especially during instances such as the COVID- 
19 lockdown. The purpose of this study is to overcome this challenge and explore the 
effectiveness of using playful techniques by incorporating elements of game design and 
robotics to teach STEM concepts in an online learning environment. This study investigates 
the effects of using playful learning in online STEM education on student engagement and 


effective learning. 


Table of Contents 


POPE OVA acsche ascites stadsienatelnccks GeubacecacosdauadcciaucscahexeushesuccssCanpabssuanduanndedewonscaacsaveesiuccaveessaresposcmunees i 
PVG GC ATION si uevessesctcssessusccscaiesecsoctd cadens stueasetpolcetuedesuasusenedesesteceasoeedl cose sniensteume desosuseedesestuasesteedeeat ii 
CErtificate OF OTIS ality siississscesssessccessospassoongsecossessuaivcessactavespeeteossouccedeipsesbasdeecstecenssceesespenses iii 
PLCKINO WIEASEMIEME <5 sccussscohsansasesedh nssseuapnosunssunacesoaibsnecssstonaesedbatsboraberasusessuesbenaccentesbbiseesersedeeede iv 
PRDSUL AC Ces ccsscivstdedasascsasseccasddesnesnscianuse edabeuspadtesneducvebesascdsauieauayscsteccebseddasneescudssseuteaecenmdecsupadstuseeses Vv 
LASE OF ADD POVIALEOMS (2 cpccsssisuadsscstanetsescextceascsnutcentecdustediustetesnddssnsuesubscuteusastebbuesconstdewnsoueedieates Viii 
WSTSU- OF DV ADIOS oscecsccsss ics seseevecsontensuctasiuseassebecassic tedaeeatonene toes veussusesncotasbouesssteneustaeueeuesctindvesseectst ix 
MEISE OF BAG UNOS siicdscexcteats teecenctasstaceussvetancoseueoteownetanendeoeeustoduvoneussonoeeseieuseadeaesuetesasesuebuuevaneunssieaeeis x 
DS SEO GUC BUM sc oo cae carats create Scce si csch eR co taes Soe tesa Sie ine cahe ua oasac cet R Os SD Sts ade caechs 1 
dl. WACK OPOUMNG. pestsicssextiecs Naish piel tana) icon dies alle altel Lamu asctadraa nds ca nich desta tie a eats 1 

DM eas « ANUS UN ALU cence ace ds cece de shines coda loca vores a etun ay aed dah deste eds ames pens Mekal 2 
Bi. 1S Y’ OVER VIC W ics, cscsdhsteincvatsdctas Mseleclovalecauls ia tnaeslaacaashaesneteas taxes duineed edu dasasbesedeuhee ns ) 

Ze “ME METAGURE RO VIC W oc. tesessscdeccsmesadaseecsdceosiiecscticcsctsandsodessottsddennussebeidasuiceonsacddccssse coesedscuseeses 4 
2o1%, ““PHeOretical, Frame WG ge tage elustonndonocunn tht al eachars aatenisaeatoaierodg ean te eagenedoniag es 4 
2.2. ROBOtICS tr STEM sisi cave easieusd snkessawvneslondiedebussavniu dev aes odes dade tava Sedebwssaauessdieeraieeteneteceead 7 
DDile WUEGO RODOICS asec eines sci casGosi ts caltcaeetaneeg eas usteStuas cea wate Raguuee Sass easeeaties 9 
Dod. . PEOUINOIRNODOICE st: acrcnsis ame adnan eee ee ip aaa ae: 9 
DD Ds: © VRITGET ROD OLECS ALG. actscet os G La Gectas ac Soa va tosui tans teasoa oom aoua grea gee Sa OE 10 

23. ° Online S71 EM environments cscs sees sarees gsiaens aie iniaeas that ut acies aes 10 
Di eWy> TCTAUC Tash diti oe aa vos asa ite suds ened ten te otudie a ned eugene sss eaas tu nibestaasda-een Sc casa a 11 
De,” KINDO AI Lae, sala’ sate teate nda ceatasus odes weemedtindy’ stasis ee vuietes cases tay eee asta Gases 15 

24. Tsering through: Play ay cai soci dain a eatin tates eens 17 
Al. Aganine Des iar FOr Wig nin 5 feces Sista ota Gs SA ek cael tek oie ol ee ed 20 

25° LdentibyineKeSGarch Gaps: sesccissiecvsassesncendendycchoaneisassinanaracaen condavoansuoaesnancunpesiassaasraes 22 
By. TRESGANCH: DESIGN cassccccsuccesnasceccctacunceesusiascatiotecansesvecsenuesndgupessucabaapbecsenabensdedstaveccobunpuauonsenate 24 
ace. Detmitic the Research: Problema was ascnpratenincrenc ne tna 24 
Bids WRESC atl Plan 662 tccheasaees tastichetaengarterne vag da Gedeons ages eantertathen aueeemenn 24 
S24... « Integrated: Desion Pramew ork is scctescce cantina eatts aaeereteg Gk anlee iene hacdanks 25 

Bis “INIGTH OD OLOGY scics cases sccuassesupceshavsacdassessaaseouaseceheassustescboasssvespussbensescaberssscesuisecstesetosctatespeunte: 27 
AL 7 “SIN GTVIEW ss te sshass aaduscccsacadapsasarotecesakes seed apen aay sadaniebaa stwccndatontasd dgneaad medendasas tans eebacateasdatenkes 27 
AD. ‘Croup: AMOCa tions cid ais bites aeaincyd Nada gua deine a No winisaaihbataitas 20 
De. VRP OUD SIZE co ata Scat te tut tet face Seed Pat et aR taal fed Sead Solel loca Sat oe et tae us Ss 21 
A! - - PPROC CAE S ss sda ciac) sudereased ins aeaviduassewnsagy axaucwadacs aia aeaeys cesta no tags an tocusaataaacoas wan an eeudanoines 28 
AAT. ARGSEARCW CUESTIONS «ccc, vcosachensescaarosaverdadanaeedaiesdeactivaneeoaen eur saned eestor eeteiran 28 


vi 


Bee: «VARIA IG Cle O UN Gs ais), cece, Senses vcieaadlsaetacene We wicasy fied eaae ese aedehea ted oemne wae yaataee, 28 

AA Ds». HAY ISOUIG SIS Ot ITN BURT. S405 fash ce es oan ces wales sade cody oouccaen cue oslee ees eles eaeaeaoees 28 
AaA Experimental StdV.: .ssccvackavetne kama eicw iad a ioenaedaneaaieaieess 29 
AAS  CONEIED SS HOI ee os ahaten ati eass eis acer ce udetet snes acl aoe tiih aloe Nit iaeul aes 30 
AAlO.< *NCOMLENE DSU VEY 2 ha% ccjuuseestaledudsls ec accc devastate ceSeccausha bledsn se cated Aawilan Ma ses ees seetnnd beans 33 
44.].. “Conducting Post-testiasc8 hsadsisince dasha sak einai eess 35 
A438. “Choosing Methods:of Anaiysissccicasieisek che Shas at eer aio 36 

Biss, AS UUIIINAT  n2255 ard chs ucaleseh Sacchaoasapecesoaaipajdass andaceydaenncaluspeed sap Sue asda asaedagdaeaandaoaavanadenaestiapeaetaes 38 
De-< DATA AMAL YSISs sy. .sccsesasvssussnnsneawbucuvendonassenedesvacssnegedenusscvancs poasaeavivensacoavasaswabecnsseuehcevandbopeies 39 
Dele, SNOW CIVIC Maes ast eacaues tyre ian wee hae et eee eased nacht hs 39 
ele” Ata COMC CTS i sistent ata sicicnalats drctarrataslal Putisd bute naginatadvcsdahaas -edesaled ivan edes stacutiels eidalntucdasaies 39 
D2y, » Part Ome: HGag Smient- Survey cccocaticiceslawss cocuccandee ndecee as vata eoseestse Se laee ae ecaares téaaees 39 
3.2.2... Part Two? Post-test Stores. 2:22.10 -hiccu viedey ei ei ead eee a ae 42 

Date. HOUMA Sec i aed at eda clea Sed ttt sh aaa ee ea Set dn aaah t poe daUetaes Be a3 
Giz. DES CUSSION oipoeié ssencasesseessdedcuenseaucensesssauesbel ccosceaecenpie lentes csasvansusedacusdesetpesshsbasconosenecunessensaeste 54 
Gly “Hinditigs or the Std y Gus ictetans Sate tesue lata ans Aa ads Ante aeons 55 
6.1.1. Relationship of Learning with Scores.............:c.sccssssssccsecsssecsssnsseecssacsesnsssnnenses =u 

ig. ANAT ACRONIS esse ait 4 deccetedavikacnpuesaicrecamnecacrasnta utara ceecaesonagacdaeraae setae erie eee 59 
Ta. MOTCIUISIO My. satessssrceseusess ts daub cterenssceadeenechatewinsoseecsbeue cs saniesuvensstealensueeusseeracsuusoevicectpndeennscos 61 
REFEREE NCES siccisccrcacosininsaavenssunssnveiadensnsaevnaetivabusonesensaeayaievenosaneauecoas ever evensawedspunsauneyatenapedeneae’ a 
PTV PCI CIC ES 5 1itesnse vases tensesseosseessbenasssuubeseatasicseevesubueos4csennsbeawasuusecsenasseuebssyusad cotatseuhsesseeiehuvencuapeeaians h 
PPT A pasha oi oe cals eoea aa trans cnianr eee oeat eae tre tian eels hugs aves ete eaia gis alae aula oust h 
|B Ba (ec lh £69) es ee ore ae reine Se One eS OE PUT ne ey NE eee ee eee Te ae? h 
PANS TNL 6 ac secrete Neila oo cs cect a Sesccartsn bgb Pas seeder was Gas dg Sacer Seal Av meee eas ie ag tee uaa RE i 
POSi-lest ENGagement SUR CY eas asks cats eacaeees ea eA aad case Sa eee i 


Vil 


List of Abbreviations 


STEM -— Science Technology Engineering Maths 
ER — Educational Robotics 

SEG — Serious Educational Games 

GBL — Game Based Learning 

EDA — Exploratory Data Analysis 


SPSS — Statistical Package for the Social Sciences 


viil 


Table 2.1: 


Table 2.2: 


Table 3.1: 


Table 4.1: 


Table 4.2: 


Table 4.3: 


Table 4.4: 


Table 4.5: 


Table 5.1: 


Table 5.2: 


Table 5.3: 


Table 5.4: 


Table 5.5: 


Table 5.6: 


List of Tables 


Constructivism’ Vs" COMStrUCthOMISM yc 5.i.sis/ex<asecencevsesdaconcqonechsacdseyodeedeaneedevdnaateeddensetes 6 
Competitors of Scratch sik isn snagyanaussiiide aden dinnmaeatieiess 14 
Chadd Wl Got: | gaspil penne PR Ott aRr ee MIR ER Ce NENT Petree ar enirin reenter or enter ren ater 24 
Participant § MEMO Ot Ap NICs: :.2,)-</as2csisaenasd shsacdsausseasesesneasaesedead deddearsanaantaqaieahiaadas 28 
COMETH VS SHOT Lecce sucess tieaes ssn eat coe aate wagered ae A oat eat ce ea cacsaue saneoneae 30 
Lesson Plan Experimental Group ss.ccsit pce enc ees ih codes descoaataneeiiaceavedetavesieattcoleesd 31 
Lesson: Plain ControlGire ap xicevcitiecntswannieasnct Gnesi Sadel acta eis a2 
Expetimental:Group Content Deliv cry aic.cac-. dest tenaseeiaad ede tans teatantn estas 34 
INGramality Ctr Grr iscsi os Gaels acleah vate daces sade oaa guceen es ab eaeee las eaten haces 43 
Normality Bxperimental Group wisi ase iutiassiaehy. ea bueno 44 
Mann-Whitney Test across Groups sscssi.isscccucssanesdoacsascasravessavacsddectunesincessnendacsiben 46 
Mann-Whitney Test across Schools ..........:ccccccsseceseceseceeseeeeeceseceeeeeeecaecneeeeeeensees 48 
Kruskal-Wallis Test across A O68 cosets seey ds hacteaseavsabettcdy ew eaees ete eneatiae 50 
Mann-Wihittey ‘Test deross GlASS acca sse ast decisueh Seated daisies 52 


ix 


Figure 3.1: 
Figure 4.1: 
Figure 5.1: 
Figure 5.2: 
Figure 5.3: 
Figure 5.4: 
Figure 5.5: 
Figure 5.6: 
Figure 5.7: 


Figure 5.8: 


List of Figures 


Integrated Design Framework of Playful Learning... cece eeeeseeseeeeeeeeeneeeees 26 
SPAUIStIC al, TCStS ots ctecslaceitanaus sleaaaas etapa Sa aasauasiiacaus ce crheu ata naw lams autem aundiaavias teas 37 
Data Analysis of Engagement Survey i.ssssvsssassacssateexessaevencobandansaiansdentarae cosas sassanes 40 
Prigdpement Ieyels across © lass scccceacccuiscujccentear enessseneimaceckaeigacesetenceeoanacenns 4] 
Engagement levels across A Ge ss iiss cesaseas Gnesuvs catantnn deasavecstasenader quaseiaaineveavancrseds 4] 
Fneagement levels:across SChOOIS jesscs, Syciepesas mettie ee eva tee aerenoeteaens Sees 42 
Manne W bitney- Test ACtOSS Groups sasiccdsisdevacesivousdents gaan ayunessivoacabe-toonnidenguentsaceanes 46 
Mann-Whitney Test across: SChoOlS:, icasinccseyguncarscsdecsscsavcacvtiv ache teaviegreaiaouatearias 49 
Kruskal-W allis “Test across A G68 5 iiis3cciscksncsashsgictvausvaaxispriccsenesva scant aecaeebteans a1 
Niann-W hittiey’ Test actos Class acon acus crite iien' suas at eaeseashiatekstsuaieat auaienes ae 53 


Introduction 


1. Introduction 


1.1. Background 

STEM is a curriculum based on the idea of educating students in four specific 
disciplines: science, technology, engineering and mathematics — in an interdisciplinary and 
applied approach. Rather than teach the four disciplines as separate and discrete subjects, 
STEM integrates them into a cohesive learning paradigm based on real-world applications. The 
aim of STEM education is to train qualified individuals to meet 21st century workforce needs. 

“STEM education is an interdisciplinary approach to learning where rigorous academic 
concepts are coupled with real world lessons as students apply science, technology, 
engineering, and mathematics in contexts that make connections between school, community, 
work, and the global enterprise enabling the development of STEM literacy and with it the 
ability to compete in the new economy (Tsupros et al., 2009).” 

Playful learning, also referred to as learning through play, is an approach that uses 
children's play to support discovery learning and problem-solving. In an online learning 
environment that aims to teach STEM education, incorporating playful learning is a challenge 
mainly because play is associated with hands on activities in a physical space whereas an online 
environment can only provide a virtual learning experience. The aim of this research is to 
overcome this challenge and explore factors that make STEM e-learning fun, engaging and 
effective, keeping the students intrinsically motivated. 

Significant research is being carried out in the field of STEM, including role of robotics 
in STEM and integrating STEM in classrooms. However, little or no research is available on 
incorporating playful learning to teach STEM concepts using robotics in an online learning 


environment. 
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1.2. Motivation 

“Generation Alpha” is considered the most technological-infused demographic up to 
date and needs to be taught and bred in an educational environment entirely different than what 
their predecessors experienced. Future workplace demands 21st Century skills, innovation and 
creativity. To achieve this purpose, students require an active learning environment in order to 
become expert learners. Pakistani students have massive potential but most lack access to an 
effective learning environment that can inculcate required skills and practical knowledge. 

The COVID-19 pandemic has created the largest disruption of education systems in 
history, affecting nearly 1.6 billion learners in more than 190 countries and all continents 
(United Nations, 2020). Closures of schools has led to home schooling and e-learning in which 
creating an effective and engaging learning environment has been a major challenge. 
Moreover, in an online environment keeping learners intrinsically motivated is a demanding 
task. Imparting knowledge online in domains that require physical learning aids such as STEM 
has been very difficult in COVID. 

We believe playful learning in online STEM will help create effective learning 
environments and benefit students. Learners will stay intrinsically motivated by learning 
through online play. It will prove to be a fine alternative to classroom education in learning 
crises such as the recent pandemic hit. The research will also pave way for future studies in 
online STEM domain. Furthermore, playful online learning can also be incorporated in schools 
to aid blended learning. Supervision, quality assurance and feedback in e-learning environment 
are also possible due to features such as recording lectures, data monitoring and tracking 
student performance. 

COVID has disrupted classroom learning; hence the main motivation behind this 


research is to study the effectiveness of an online learning environment where students can 
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playfully learn STEM skills as it is essential in making Pakistani learners become innovators, 
scientists and makers of tomorrow. 
1.3. Study Overview 

The purpose of this study was to minimize the gaping hole in the research world by 
exploring the effectiveness of using playful techniques by incorporating robotics as a learning 
aid to teach STEM concepts online. It aimed at investigating the effects of using playful 
learning in online STEM on student engagement and effective learning. 

The design framework implemented for this study describes the types of engagement 
created through game design elements to foster playful learning. For this research, we have 
focused on studying the impact of affective, cognitive, and physical engagements of students 
created by the inclusion of game design elements in a playful online learning environment to 
teach a STEM topic. 

As this study was conducted online, elements of game design were implemented in the 
research by using a robotics website to attain playful learning. Students had to code their virtual 
robots through block-based programming and solve certain challenges, this helped incorporate 
the design framework as well as the STEM topic chosen for this study. This research was an 
attempt to compare playful pedagogy with traditional teaching methods in online learning 
during the lockdown period in COVID. 

A quasi experimental study was conducted and non-equivalent control group post-test 
only design was employed for this research. Students of Grade 6 and 7 from Army Public 
Schools of Rawalpindi and Nowshera were chosen for the control and experimental groups 
with 50 participants in each group (total sample size N=100). 

A significant difference in the means of both the groups was revealed when quantitative 
data analysis was performed. Therefore, it was concluded that using robotics and game design 


elements to implement playful learning results in enhanced engagement of students. 
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2. Literature Review 


“We live in a world that is changing more rapidly than ever before. Today’s children 
will face a continual stream of new issues and unexpected challenges in the future. Many things 
that they learn today will be obsolete tomorrow. To thrive, they must learn to design innovative 
solutions to the unexpected problems that will undoubtedly arise in their lives. Their success 
and satisfaction will be based on their ability to think and act creatively. Knowledge alone is 
not enough: they must learn how to use their knowledge creatively” (Resnick, 2014, p.13). In 
traditional school classrooms, it is observed that students who have short attention spans often 
display great concentration when engaged in projects that interest them. Seymour Papert has 
found that learners become deeply engaged by “hard fun”. Simply put, learners do not mind 
activities that are difficult if the activities connect deeply with their interests and passions. 

Evidence suggests that improved student engagement is a result of capturing students’ 
attention, nurturing deep learning, and minimizing cognitive load ultimately leading to better 
educational outcomes (Mulqueeny et al., 2015). 

In classrooms, teachers can use robotics activities to integrate STEM and incorporate 
21* century skills, including creativity (Marghitu et al., 2012), collaboration, critical thinking 
(Freiman et al., 2010), computational thinking (Atmatzidou & Demetriadis, 2016; Keane et al., 
2016), and communication skills (Prelock & Nelson, 2012). 

2.1. Theoretical Framework 

The robotics learning environment is strongly associated with two central learning 
theories that have been influencing science education since the mid-twentieth century: 
constructivism and constructionism. Constructivism is a theory about how people learn (Piaget, 
1965). It suggests that learners construct knowledge out of their own experiences. Based on 
Piaget’s theory of constructivism, Papert (1980) proposed the constructionism learning theory, 


according to which meaningful learning takes place when students construct real physical 
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objects that are meaningful to them and that they can share with others. Based on the 
philosophy of constructionism, tangible tools can be used to ‘think with’ in order to explore 
STEM concepts (Barker & Ansorge, 2007). The hands-on applications of STEM concepts help 
develop 21st century skills, as objects such as robots are constructed to explore and experiment 
with ideas (Mikropoulos & Bellou, 2013). 

The theories of constructivism and constructionism offer a framework for developing a 
learning environment in which individuals learn by making, creating, programming, 
discovering and designing their own objects that allows them to think with. For example, 
teachers allowing kindergartners to play with Play-Doh to make objects meaningful to them 
can be an example of a constructivist approach. Whereas, students working collaboratively on 
a project to build a plant watering robot facilitated by their teacher could befit the 
constructionism learning theory. 

Rob & Rob (2018) define the role of teacher and learning environments pertaining to 
both theories as follows: 

Constructivist Teacher: A constructivist teacher sets up the environment that fosters 
individual learning for students and presents a problem to be solved, while the students go on 
their own way to produce a personally meaningful artefact without any further teacher’s 
intervention. The learning environment can be termed as teacher initiated. 

Constructionist Teacher: A constructionist teacher sets up the environment for 
collaborative learning of students, then he or she defines the problem to be solved and the 
meaningful product to be developed, and then guides them to reach towards the goal. The 
learning environment can be termed as teacher facilitated. 


Hence, we can associate the learning theories with the attributes outlined in Table 1. 
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Table 2.1: Constructivism vs Constructionism 


Theories Attributes 
Experience Artifact Collaboration Use of Tools, 
Media and 
Context 
Constructivism | Knowledge is Construct a | Not necessary Not necessary 
inherited from _| Personal 
past experience | Artifact 
Knowledge is 
constructed by 
new experience 
Constructionism | Knowledge is Construct a | Collaborative Learning is 
inherited from | Meaningful | Creation Process | enhanced by use 
past experience | Artifact of tools 
Sharing of (computer, pen) 
knowledge is artifacts Media (face to 
constructed by face or online 
new experience classroom) and 
Context (real 
world product, 
educational goal) 


Robotics, with its multi-disciplinary nature, provides constructive learning 
environments that are suitable for a better understanding of STEM subjects (Khanlari, 2014). 
Playful approaches in education sit within a constructivist theory of learning. This can also be 
associated to gamification of educational content and learning through games. Gamification 
has emerged as a common instructional intervention to improve learner motivation and learning 
outcomes. 

According to Foreman, (2003, p.15), ‘‘Games expose players to deeply engaging, 
visually dynamic, rapidly paced, and highly gratifying pictorial experiences that make almost 
any sort of conventional schoolwork (especially when mediated by a lecture or text) seem 
boring by comparison.’’ The ability to take on multiple roles allows for the game player to gain 


multiple perspectives of a given scenario which facilitate richer construction of knowledge and 
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deep integration of content. Often referred to as “‘guided discovery’’, learners immersed in 
well-designed Serious Educational Games (SEGs) are able to monitor their own thinking 
(meta-cognition), gain new knowledge, and revise existing schemata. 
2.2. Robotics in STEM 

A review of relevant STEM education literature highlighted that students were more 
likely to participate in STEM activities if they had teachers who provided engaging STEM 
activities in the classroom (Hayden et al., 2011). Robotics has proven to be an engaging tool 
for motivating students to participate in STEM activities (Ludi, 2012). Robotics has also proven 
to be an effective tool to engage and stimulate teachers’ interest in STEM learning and teaching 
(Lee et al., 2014). Robotics-based professional development can assist teachers with 
developing skills in designing, constructing, and programming and with developing an 
understanding of how constructionist approaches to learning can enhance problem solving and 
higher-order thinking among their students. 

Robotics activities provide great opportunities for students to learn important mathematics 
and science concepts—and learn them in a much more meaningful and motivating context than 
in traditional classrooms (Mitchel Resnick, 2004). Robotics can be especially effective in 
teaching STEM, as it enables real-world applications of the concepts of engineering and 
technology and helps to remove the abstractness of science and mathematics. In fact, various 
robotics activities led to improvements in science, technology, engineering, and/or 
mathematics learning (Kim et al., 2015). According to Karim (2016), Robots have the potential 
to be an effective add-on to traditional education. Robots bring excitement and the opportunity 
to experiment with tangible objects in classroom which is considered effective for learning. 

ER4STEM was a project launched that aimed to realize use of educational robotics 
creatively to maintain children’s curiosity in the world. This would lead them to 


entrepreneurial, industrial and research careers in STEM fields, by exploiting the 
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multidisciplinary potential of robotics with a well-structured interdisciplinary approach 
(Lammer et al., 2017). 

Robots spark the imagination of children and adults alike since they are relatable to 
humans in doing different tasks, this can be attributed as a reason to growing use of Educational 
robotics (ER). ER is often presented as a platform for teaching STEM, developing skills such 
as scientific inquiry, engineering design, problem solving, creative thinking and teamwork. ER 
enhances motivation and contributes to cooperation, confidence, and creativity. Therefore, ER 
applications also have a potential to improve classroom teaching (Kucuk & Sisman, 2020). 

Robotics allows students to learn about STEM’s interdisciplinary nature, encouraging 
students to work collaboratively (Yuen et al., 2014). In a study that examined elementary and 
middle school students’ participation during a summer camp, robotics was found to be highly 
engaging (Yuen et al., 2014). Additionally, programming robots helps students to engage in 
science inquiry as described by Linn and Hsi (2000): (a) science is made accessible by engaging 
with physical models; (b) thinking is made visible through construction and design principles; 
(c) students learn from each other through collaboration; and (d) autonomous learning skills 
are developed through self-directed learning. Whether it is through the engaging process of 
constructing robots or the excitement of the competitions, several robotics programs have 
resulted in an increase in students’ comfort level with applications of STEM, development of 
twenty-first-century skills, and increased interest in pursuing STEM-related programs beyond 
high school (Brand & Kinash, 2013; Grubbs, 2013; Hunter et al., 2006). 

Physical robots can embody mathematical concepts allowing greater affective 
engagement. Virtual robotics facilitate the planning and modelling of physical robotics 
encouraging expression of symbolic mathematical language (Samuels & Haapasalo, 2012). 

The interaction with robots increases motivation, engagement and attitude towards 


education. “Admittedly, with the simplification of robot design and assembly process; the 
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inclusion of intuitive visual drag and-drop programming and the gradually reducing cost of 
educational robot platforms, we are experiencing the advent of a new era in educational 
technology” (Karim et al., 2016) 

2.2.1. Lego Robotics 

Lego robotics is used as an authentic and kinesthetics way to improve children’s 
problem-solving skills, reinforcing science applications and concepts, while building upon 
informal learning activities often done at home (Karp & Maloney, 2013). 

Using Lego EV3 robotics builds spatial visualization skills as students manipulate Lego 
pieces to build the robot. Other mathematics skills include proportional reasoning as students 
calculate wheel rotation and approximate the distance the robot will travel once it is 
programmed (Grubbs, 2013). “‘Not only does the design and hands-on component of 
technology stay intact, but students also enjoy solving a realistic engineering scenario, role 
playing...and constructing physical components through an open-ended design challenge’’ 
(Grubbs, 2013, p.12). 

Lego Mindstorms is a hardware and software structure which is produced by Lego for 
the development of programmable robots based on Lego building blocks. Research has found 
that introducing Technology with Lego Mindstorms to young kids is more effective than 
making them code early on. Even in children as young as 6 years old, Robotics with Lego is 
found effective to expose them to both the principles and ideas of coding, like logic, and 
elements of engineering through robotics. The small parts in Lego Mindstorms challenge 
younger students developing motor skills and coordination. (Learning through Play with Lego 
Mindstorms — BSD Education, n.d.) 

2.2.2. Arduino Robotics 

Arduino-based educational robotics can promote a collaborative development 


environment that helps to develop innovation and student motivation during the learning 
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process. When it comes to young learners, it is preferable to start from simple challenges and 
increase the difficulty gradually. The combination of theory and practice is important, linking 
fun tasks with challenges by applying theory to problem solving aids learning. Arduino is a 
tool to consider in this regard, since Arduino software is free and compatible with most 
operating systems (Plaza, Blazquez, et al., 2018). 

Arduino is a software and hardware solution that requires greater skill on the part of 
students. This is because the programming is textual, and the hardware components used are 
not intended for use by young students. Despite this, Arduino allows applications to be 
implemented without any limitations (Plaza et al., 2019). 

Some Arduino Robotics kits include Elegoo Smart Robot Car Kit, Makeblock 
Educational robot kit, Codibot, Arduino Robot etc. 

2.2.3. Other Robotics kits 

Apart from Arduino and Lego kits, there are other Robotics kits available as well that 
aim to teach STEM. Some of these include, “Robo bit buggy” based on micro:bit 
programmable controller. “GoPiGo 3” is a robot car based on Raspberry Pi. “Sunfounder 
PiCar” is also a kit on Raspberry Pi. “Cozmo” is a commercial robot that has a beginner- 
friendly interface and is deemed a great educational robot for kids and adults to learn to code 
creatively. “Meccano Meccaspider” robot kit comes with 291 building parts and can be 
controlled via Meccano educational app. 

2.3. Online STEM environments 

Web based applications promote learning in many ways overcoming the limitations of 
time and physical space. Owing to the need of media literacy as an essential 21st century skills, 
many online platforms provide the opportunity for online education. Some online environments 


focusing on STEM education are mentioned as follows: 
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e “VEXcode VR” lets you code a virtual robot using a block-based coding environment 
powered by Scratch Blocks. 

e The “Open Roberta® Lab” is a freely available, cloud-based, open source programming 
environment that makes learning programming easy - from the first steps to 
programming intelligent robots with multiple sensors and capabilities. 

e “simulator.io” is an online CAD tool for logic circuits. Easiest way to learn how to build 
logic circuits. 

e “Tinkercad” is a free, online 3D modelling program that runs in a web browser, known 
for its simplicity and ease of use. Since it became available in 2011 it has become a 
popular platform for creating models for 3D printing as well as an entry-level 
introduction to constructive solid geometry in schools. Its add on, Tinkercad circuits 
allows wiring of basic electrical circuits as well as programming circuits integrated with 
Arduino and testing simulations of circuits. 

e “BrainPop” is a group of educational websites with over 1,000 short animated movies 
for students in grades K-12, together with quizzes and related materials, covering the 
subjects of science, social studies, English, math, engineering and technology, health, 
and arts and music. 

Two of the most popular and STEM focused online platforms are Scratch and 
Minecraft. For this thesis, these two platforms have been reviewed in detail and outlined in the 
headings below. 

2.3.1. Scratch 

Mitchel Resnick believes that in order to help young people prepare for a world that is 
changing more rapidly than ever before, it is important to embed making and coding in a 
creative-learning process characterized by Projects, Peers, Passion, and Play; namely the 4P’s 


which form the design principles of the Scratch learning environment. 


11 


Literature Review 


With Scratch, young people can design their own interactive media including stories, 
games, animations, and simulations by attaching coding blocks together, “just as one might 
snap together LEGO bricks or puzzle pieces” (Brennan et al., 2009). 

The Scratch programming environment and language provide users the opportunity to 
learn programming very quickly and it has variety to keep learners engaged for years (Maloney 
et al., 2010). 

Similar to Scratch online environment is another tool, Crumble that combines block 
programming software with hardware and allows students to make electronic assemblies in a 
simple way but for beginner coders, Scratch is recommended before they can start using 
Crumble, followed by Arduino. As Scratch is an ideal tool for children and adults to learn with 
no prior programming or robotic experience (Plaza, Sancristobal, et al., 2018). 

2.3.1.1. | Robotic kits with Scratch 

Research has found out that many hardware components can be integrated with scratch 
which allow easy programming as well as provide the opportunity to have a physical hands-on 
learning experience. 

Lego Robots can also be programmed in the Scratch environment. “Lego WeDo” 
Construction Kit is a simple robotics tool designed for ages 7—11. It allows users to design their 
own interactive machines, and then program them using drag-and-drop coding blocks. The 
Lego WeDo Construction Kit can be used with the online editor of Scratch 2.0 by adding an 
extension. The robotics kit connects over Bluetooth to Scratch platform. Combined use of 
Scratch and WeDo in robotic projects falls under constructionism learning. These two 
platforms greatly facilitate introduction of programming and robotics in primary school (Olabe 
et al., 2011). To connect “Lego Mindstorms NXT” to Scratch, Enchanting is a modification 


that is used. “Lego Mindstorms Education EV3” can also be connected to scratch by enabling 
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an extension inside the Scratch environment expanding possibilities and allowing learners to 
build robotic puppets and tell stories, make musical instruments and game controllers etc. 

“Finch” is a small simple robot designed by Carnegie Mellon’s CREATE lab. It is 
connected by USB and requires Bird brain robot server. It has support fora range 
of programming languages and environments and can be integrated with Scratch. 

Similarly, “Hummingbird” is a Robotics kit that comprises of lights, sensors and motors 
which allow students to build a robot out of any material and needs Birdbrain server to run. 
Hummingbird epitomizes a low-floor, high-ceiling approach; it is suitable for beginners and 
allows learning advanced engineering and programming as well. 

The “micro:bit” is a tiny circuit board designed to help kids learn to code and create. It 
can be programmed in Scratch by enabling an extension inside the Scratch platform. 

Furthermore, “S4A” is a Scratch modification that allows simple programming of 
the Arduino hardware platform. It provides new blocks for managing sensors and actuators 
connected to Arduino. “Makeblock” is a similar programming environment and allows Scratch 
visual programming language with Arduino. 

The “GoPiGo3” is a Raspberry Pi Robot Car kit that is easy to build and program with 
Scratch, Python, Blockly, and other programming languages. 

“ScratchJr” is a visual programming language designed to introduce programming 
skills to children ages 5—8. By creating projects in ScratchJr, young children can learn to think 
creatively and reason systematically, despite not being able to read. Some commercially 
developed educational robots that can be integrated with Scratch and Scratch Jr are mentioned 
below (Mickel, 2015): 

e Dash and Dot are robots produced by the California based start-up Wonder Workshop. 


The robots, intended for ages five and up, are fully assembled and available for 
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purchase as a set for $230. ScratchJr extension connects to Dash and Dot using their 
proprietary API via Bluetooth. 

e Sphero and Ollie, created by Orbotix, are robots that drive around, change colour, and 
do various tricks, programmed in ScratchJr to drive and change colour. 

e Light Play (and AdaFruit BlueFruit LE) project explores kids' experimentation and 
tinkering with light, colour, and shadows. ScratchJr extension to enable 
programmability of the direction and speed of the turntable, as well as the colour and 
intensity of the lights. 

e KIBO, a programmable robot for children 4-7 years old ($229), is produced by 
KinderLab Robotics and Marina Bers (a collaborator on ScratchJr). The current version 


of KIBO is programmed by scanning wooden blocks (with barcodes). 


Table 2.2: Competitors of Scratch 


Platform Age Coding Build | Gamification | Hardware | Free 
Games integration 
Block | Textual 
Snap 12+ | Yes Yes Yes Yes 
Blockly 8+ Yes Yes 
Stencyl IZ || Yes Yes Yes 
App 10+ | Yes Apps Yes Yes 
inventor 
Game 13 Yes Trial 
Salad 
Tynker a+ Yes Yes Yes Trial 
Code.org K- Yes Yes Yes Yes Yes 
12 

Code K-8 | Yes Yes Yes Trial 
monkey 
Code 8+ Yes Yes Yes Trial 
Kingdoms 
Code Gr Yes Yes No 
Combat 5+ 
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Alice Gr Yes Yes Yes Yes 
6+ 
Kodable 4-10 | Yes Yes Yes Trial 


2.3.2. Minecraft 

Minecraft is an open-ended game where players gather resources and build structures 
by destroying and stacking 3D blocks in a virtual world. It is among the most popular video 
games of all time with over a 100 million registered users (Makuch, 2014). 

Minecraft has been adopted by educators worldwide for educational purposes (Schifter 
et al., 2013). Its rise and popularity can be attributed to the reason that interactions in Minecraft 
involve a broad range of educationally relevant content compatible with classical and modern 
theories of learning (Lane et al., 2017b). 

Additionally, Minecraft for Education Edition (MinecraftEdu) is successful as it offers 
engaging educational content with variety of features designed to help students learn the basics 
of programming and apply coding on STEM subjects. It also has a flexible curriculum 
contributing to the development of computational thinking skills. It is possible to discover 
conditions, functions, coordinates and coding concepts with Minecraft for students of different 
ages (Minaudo, 2020). 

Taking programming to a further gamified approach, “Hour of Code” Minecraft 
supports learning with engaging game-like graphics, a model world revolving around a 
storyline, tutorial instructions and a block-based programming language like Scratch. 
(Ghasemaghaei et al., 2017). 

Minecraft is being used by educators in STEM subjects. In Science it is used to show 
how atoms become stable and unstable and to illustrate the features of cells. In Mathematics, 
it helps students visualise mathematic equations. For educators in language, history and arts, 


Minecraft can be used to help students experience historic locations, artistic periods and 
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architectural traditions (Brand & Kinash, 2013). Building a functioning clock from scratch in 
Minecraft requires an understanding of circuitry, the ability to make the appropriate 
calculations, and the ability to craft and design a model. Therefore, building a clock would 
relate to electrical engineering, mathematics, and mechanical engineering (Lane et al., 2017a). 

Students working with Minecraft are unafraid to try a different configuration, to make 
a new tool, or to discover the attributes of a stone. Their only limitations may be what questions 
to ask and which problems to solve, and that is where the teacher contributes meaningful 
scenarios and pertinent questions reflective of the curriculum (Bos et al., 2014). 

Minecraft’s state and action space allow users to create complex systems, including 
logic gates and functional scientific graphing calculators. It can serve as a platform to model a 
wide range of robotics tasks such as cooking assistance, assembling items in a factory, object 
retrieval and complex terrain traversal (Aluru et al.,2015). 

2.3.2.1. Robotics with Minecraft 

Minecraft also provides opportunities to players to learn and experiment with robotics. 
“Code Builder” is a tool that shows up in the Minecraft game as a robot. Users can interact 
with the game through the robot agent via learn-to-code platforms such as Tynker, Scratch and 
MakeCode using which players program actions that the robot performs in the game, just like 
a physical robot would execute instructions. This allows players to experience in-game virtual 
robotics. Another platform, ‘“ComputerCraftEdu” brings programmable turtle robots 
to Minecraft. Players start with a tile-based interface to learn the fundamentals of programming 
in a fun, accessible environment. They get engaged and motivated to find ways to use turtles 
to automate and extend their usual Minecraft activities using this software. ComputerCraftEdu 
comes included in MinecraftEdu but can also be downloaded separately for both MinecraftEdu 


and regular Minecraft. 


16 


Literature Review 


Extending Minecraft to physical robotics, some projects have been made using 
Arduino, Raspberry Pi and Lego NXT in Minecraft by keen learners including Minecraft 
controlled Raspberry Pi Robotic arm (Robots and Physical Computing: Minecraft Controlled 
Raspberry Pi Robot Arm, n.d.). A tool “MCreator Link” enables users to connect hardware 
devices such as Arduino and Raspberry Pi with Minecraft game via McCreator 
procedures, commands and general API for Minecraft mod developers. 

2.4. Learning through Play 

“Informed Learning is when children learn through their own exploration and activity, 
as they follow their personal interests and grasp concepts naturally. Also referred to as playful 
learning,” (Playful Learning - Delaware Children’s Museum, n.d.). “Playful learning, also 
referred to as learning through play, is an approach that uses children's play to support 
discovery learning and problem-solving” (Playful Learning | Early Learning Toolkit, n.d.). 

Learning through play is a widely explored approach to learning and teaching and has 
been used for children’s learning. Research into playful learning approaches relating to adults 
is more limited. Defining play or playful behaviour is difficult to do as it contains many context 
dependent qualities. However, Play can be defined as an experience; it has intrinsic rather than 
extrinsic motives; the process is more important than the outcome and it involves some level 
of active engagement (Henricks, 1999). 

Learning through play is a critical element for young children to develop key skills in 
language, emotion, creativity and social interaction, it pulls together the logical and creative 
areas of the brain that result in intrinsic motivation to perform a task. 

In the world of new technologies, the forms of education are changing, as are the sites 
on which education occurs. Some scholars argue that the school of the future should be based 
predominantly on innovation and interactive creativity with new technology and new ways of 


acting (e.g. Natriello, 2007; Sawyer, 2006; Tuomi, 2007). This aim has been widely recognised 
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globally in educational contexts, and it has been concluded that formal teaching approaches 
may not match the learning methods used by children and young people (Sawyer, 2006). It also 
is argued that many students learn to solve specific types of problems but are unable to respond 
to unexpected situations, which inevitably arise in today’s fast-changing world (Resnick, 
2007). As Resnick (2007) and Sawyer (2006) assert, most schools are not focusing on helping 
pupils develop as creative thinkers and are not teaching them to create knowledge. Instead, in 
formal schooling, children are typically taught that knowledge is static and complete; they 
become experts at consuming knowledge rather than producing and creating it. Thus, 
innovative ways of using information and readiness to deal with the unexpected demands 
importance (Kangas, 2010). 

Play has its value as a mean in children’s learning. As humans are born curious, it is 
but a natural way to learn because it uses all senses to solve problems and understand the 
environment. Play prepares children for academic learning as they begin their school years and 
each step along the way (Rapeepisarn et al., 2006). 

To give theoretical construction to the term playful learning, we will define it in the 
context of ‘magic circle’. The term ‘magic circle’ was originally coined by Huizinga (1955) as 
an example of a space in which play happens. It is the space in which the normal rules and 
reality of the world are suspended and replaced by the artificial reality of a game world. Within 
this definition of the magic circle, there are three key pedagogical characteristics for using 
playful approaches to learning in higher education: the positive construction of failure; support 
for learners to immerse themselves in the spirit of play; and the development of intrinsic mo- 
tivation to engage with learning activities (Whitton, 2018). This helps build resilience to failure 
and fosters the ability for students to take measured risks, making them innovators focusing on 
learning challenges rather than the measurable outcomes (Dweck, 2010). This ability to enter 


a parallel fantasy world sets the imagination free and leads to greater creativity and innovation 
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(Bateson & Nettle, 2014). Activity within the magic circle is by choice and intrinsically 
motivated. Participation is motivated by the internal benefits of play, rather than for any 
external rewards. Intrinsically motivated engagement allows for personal exploration, 
experimentation and discovery, leading to personally meaningful learning (Whitton, 2018). 
Techniques that encapsulate elements of play involve role play or storytelling, aspects 
such as making, building or tinkering that involve playful experimentation and learning through 
errors and making mistakes. They also include playful ways of framing challenges to support 
motivation, such as the use of problems or quests. These types of challenge create motivation 
for people to engage and stimulate curiosity and feelings of satisfaction on completion. 
Playful approaches to learning can be an effective mode of teaching and learning 
requiring activity and sensation, but experience alone is not always enough to achieve learning. 
There must be some critical reflection to turn the experience into learning (Rice, 2009). The 
inclusion of play helps to generate excitement, enjoyment and interest as part of the process of 
learning. It can be understood as part of the VARK learning approaches, which help with 
motivation, engagement, and allow different learners to approach a subject from different 
perspectives. To conclude, playful learning requires a shift from students being passive 
consumers of knowledge towards active creators of knowledge. Primarily for this research, we 
will define playful learning as an activity by the learner, aimed at the construction of a mental 
model, that is designed to include one or more elements of games for the purpose of enhancing 
the learning process. Depending on learning goals, learners, settings, and other factors, 
designers conceptualize and implement playful learning environments that are either games, or 
that incorporate game-like elements (Plass et al., 2014). We will henceforth use the term playful 
learning to describe learning that incorporates game elements and explore further on that 


subject below. 


19 


Literature Review 


2.4.1. Game Design for Learning 

Some of the fundamental elements of game design include game mechanics, visual 
aesthetic design, narrative design, incentive system, musical score, content and skills (Plass et 
al., 2014). The motivational function of games is their most frequently cited characteristic. 
Motivation and engagement of young pupils are sought through the gamification of the learning 
process (Kintsakis & Rangoussi, 2017). 

During the research intervention by Kintsakis & Rangoussi (2017) pupils were found 
motivated and engaged, thanks to the graphics, the look and feel of the environment and the 
gaming elements. Hence, it is found that multimedia elements (animation, graphics, video, 
frames) greatly facilitate and enhance the reception of subject matter. 

Using games effectively during instruction encourages students to actively participate 
in their own learning (Leonard et al., 2016). A variety of gaming strategies have been employed 
in education such as game design and programming (Webb, P. I. & Pearson, 2012), mobile 
games (Koutromanos & Avraamidou, 2014), online games (Mysirlaki & Paraskeva, 2010) and 
game authoring tools (Robertson & Howells, 2008). 

Studies related to gaming have demonstrated development of a more positive attitude 
and motivation toward mathematics (Ke, 2008). Ke (2008) conducted a study that explored the 
effect of using a series of web-based games called ASTRA EAGLE on cognitive mathematics 
accomplishment, metacognitive awareness, and favourable attitudes toward mathematics 
among elementary students in Pennsylvania. Results revealed that children developed positive 
attitudes toward learning mathematics through gaming. Digital game playing has also been 
used successfully to teach mathematics problem-solving (Chang et al., 2012) and can be used 
as a social practice to support the development of ‘‘strategic thinking, planning, 
communication, application of numbers, negotiating skills, group decision making, and data- 


handling’’ (Li & Ma, 2010). Incorporating computer games in the mathematics classroom has 
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been shown to lead to favourable attitudes toward learning mathematics and to increases in 
mathematics achievement and student success. 

Serious Educational Games (SEGs) can bring about a life-like experiences and assist in 
the creation of problem-based learning that cannot be replicated in the traditional science 
classroom or even on field trips (Lamb et al., 2015). Furthermore, SEGs have shown positive 
gain-scores after science students played teacher created games. According to a study, science 
students who participated in a game based activity displayed more levels of engagement than 
students who participated in non-game based activity (Annetta, 2010). 

The research literature tells us that twenty-first century skills include media-based 
intangibles often found in video games. The ideas of play, performance, multitasking, 
distributive cognition, networking and negotiation are not only common in games but have 
become some of the common characteristics found in today’s school age students (Annetta et 
al., 2013). 

Ricci et al. (2014) proposed that instruction that incorporated game features enhanced 
student motivation, which led to greater attention to training content and greater retention. 
Thornton & Cleveland (1990) noted that the essential aspect of a game is interactivity. Gredler 
(1996) stated that the essential elements are a complex task, the learner’s role, multiple paths 
to the goal, and learner control. Malone (1981) argued that there are four characteristics of 
games that make them engaging educational tools: challenge, fantasy, complexity, and control. 

Modern Game-Based Learning (GBL) approaches are commonly build on applications 
that have defined learning outcomes and are designed to promote active participation and 
interaction, balancing the learning objective with the gameplay, in order to enhance the ability 
of the learner to retain and apply the knowledge gained to the real world while being completely 
involved and thus more receptive. SEGs often exploit narratives, storylines, visual elements 


and other features common to entertainment games, such as scoring and social networking in 
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order to motivate and engage players in a learning activity. Educational games have learning 
goals and structure, but in addition are adaptive and interactive and most importantly they 
provide enjoyment, pleasure, motivation, gratification and emotion, in order to achieve learner 
engagement and involvement. Games create simulated environments that facilitate immersion, 
allowing learners to explore alternative approaches to situations virtually, in order to directly 
experience practical and emotional consequences of their actions. Through these approaches, 
GBL allows learners to directly though virtually, experience the real-world by developing their 
awareness of real situations (Protopsaltis et al., 2010). 
2.5. Identifying Research Gaps 

As digital creation tools become more prevalent, early childhood education remains an 
area in which few educational technologies focus on digital creation or high-level thinking 
(Flannery et al., 2013). It is not the hardware or the software, but the combination between the 
use of the new interactive technologies and the pedagogical approach that has the possibility 
to facilitate significantly young children’s learning achievement (Papadakis et al., 2016). 

Researchers have explored robotics as a platform for teaching mathematics, physics, 
programming and engineering design and problem solving. Yet, only some of these studies 
defined exactly the specific knowledge of mathematics, science and technology the student 
should learn in a robotics program or evaluated how robotics activities contributed to students’ 
learning in these subjects (Highfield, 2010). 

Physical contact in the classroom is generally thought to be superior in developing and 
maintaining motivation and engagement of the pupils, as well as in achieving the desired 
learning outcomes. The suitability of e-learning for younger ages and especially for Primary 
Education, therefore, is still an open issue, GBL, or even the mere gamification of the learning 
process, is a valid method to achieve motivation and engagement across all grades of education, 


even more so for younger pupils. How this method can be utilized for early ages specially in 
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STEM still needs research and motivation through gamification is usually extrinsic in nature. 
How intrinsic motivation can be achieved through gamification is still a question. 

ER is still frequently associated with teamwork and problem-solving development, 
extracurricular activities, and Lego robots (Khine, 2017). Robotics projects are more 
commonly associated with technology and engineering. There is limited focus towards science 
and maths. ER activities are more focused towards information technology and not STEM 
(Kucuk & Sisman, 2020). 

In school reality, teachers are the key factor to the application of Robotics Education. 
More investigation is also required on how teachers could use better these digital tools. 
(Dorouka et al., 2020). There is significant scope to enhance the trust levels of teachers and 
students in the robotics for STEM education (Hu & Garimella, 2015). 

The research of the learning effect of games is still in the beginning, but some studies 
already show the potential of games such as students staying more motivated when learning 
with a game (Garris et al., 2002). 

The extensive literature review suggests that significant research is available in the field 
of STEM, including integration of robotics and online learning platforms such as Scratch and 
Minecraft in classrooms to teach STEM. However, there is scarcity of research available on 
incorporating playful learning approach to teach STEM concepts to young learners in an online 
learning environment; using robotics or games that keep the learner intrinsically motivated and 


result in effective learning outcomes. 
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3. Research Design 


3.1. Defining the Research Problem 
The literature review suggests that research to teach STEM concepts online using 
playful learning approach is insufficient. Furthermore, imparting knowledge online in domains 
that require physical learning aids such as STEM has been very difficult in COVID. Thus, to 
create an engaging e-learning environment that results in effective learning outcomes and keeps 
the learners intrinsically motivated, further research is required. 
3.2. Research Plan 
Playful learning in higher education currently lacks a coherent definition, evidenced 
pedagogic rationale or framework of implementation approaches. Playful learning in higher 
education is an emerging form of practice that includes many types of playful approaches to 
teaching, learning, research and academic practice. These approaches facilitate positive 
attitudes towards failure, embody a spirit of play and experimentation, and enhance intrinsic 
motivation. Whitton (2018) categorised these playful approaches into three distinct groups: 
playful tools, playful techniques, and playful tactics. Outlined below in table are playful 
learning tools, techniques and tactics (Whitton, 2018). 


Table 3.1: Playful Learning 


Playful Learning Description Examples 
Tools Objects, artefacts and Games 
technologies that signify a Toys 
playful environment Simulations 
Puzzles 


Virtual environments 


Techniques Pedagogies and learning Role play 
approaches that facilitate Making 
play Performance 

Problems 
Quests 
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Tactics 


Mechanics and attributes 
that engender playfulness 


Surprise 
Humour 
Chance 
Competition 
Storytelling 
Mystery 
Badges 


For our research, we aim to incorporate the relevant tools, techniques, and tactics 


through an integrated design framework of game-based learning to attain playful learning 


thus fostering engagement of learner. 


3.2.1. Integrated Design Framework 


The framework we have chosen to implement for this study is proposed by Jan L. Plass, 
Bruce D. Homer and Charles K. Kinzer. It describes what kinds of engagement playful learning 


environments allow and defines the game design elements that create such engagement (Plass 


et al., 2014). 
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Playful Learning 


és A 


Affective Physical Cognitive Social/Cultural 
Engagement Engagement Engagement Engagement 


Learning Game Design Elements 
Knowledge/Skills ¢ Incentive System ¢ Learning Mechanics ¢ Assessment 
Mechanics ¢ Aesthetic Design ¢ Narrative ¢ Musical Score 


Affect Cognition Social / Cultural 


Motivation & Interest Situated in Context Social Context 


Emotional Design: Transfer of Learning Participatory Learning 
Representation Culture 
Scaffolding & Feedback 
Emotional Design: Social Aspects of Agency 
Interactions Dynamic Assessment 


Observational Learning 
Goal Orientation Information Representation 
Relatedness/Self 
Self-Efficacy Meaningful Interactions Perception 


Self-Esteem Gestures & Movement Social Interaction Design 


Figure 3.1: Integrated Design Framework of Playful Learning 
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4. Methodology 


4.1. Overview 

The basic idea of this study was to implement playful learning techniques to teach 
STEM concepts and check if it impacted engagement levels for students in an online 
educational environment. The topic chosen was ‘speed, distance and time’. A Quasi- 
experimental study was conducted. Post-test only control and experimental group design was 
employed. 

4.2. Group Allocation 

Since the purpose of this research was to compare if using playful approach to learning 
impacted the engagement levels of students for teaching STEM concepts in an online setting, 
the quasi experiment was conducted on students ranging between ages 11 to 14. 

Students of Grade 6 and 7 from Army Public Schools in Rawalpindi and Nowshera 
were chosen for the control and experimental groups. 

To set up an environment for a fair comparison, students of Grade 6 and 7 were chosen 
from Army Public School Fort Road, Rawalpindi and Army Public School Iqbal, Nowshera. 
These students were then divided into control and experimental groups. Choosing different 
locations allowed to cater for diversity of learners. To ensure uniformity, all the sections were 
taught the same topic by the researcher, keeping pedagogy different for control and 
experimental groups. Learning objectives pertaining to the topic were kept same for all 
sections. The topic ‘speed, distance and time’ was taught using an applied interdisciplinary 
approach using examples from daily life to ensure underlying STEM concepts are conveyed to 
both the groups. 

4.3. Group Size 
The control and experimental groups comprised of 50 participants each. Table 


summarized the demographics of the participants. 
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Table 4.1: Participant's demographics 


Categories Schools Grade Group 


APS Fort Road | APS Iqbal | Grade 6 | Grade 7 | Control | Experimental 
(Rawalpindi) | (Nowshera) 


No. of 61 39 54 46 50 50 
Participants 


4.4. Procedure 

4.4.1. Research Questions 

This study was an attempt to compare playful pedagogy with traditional teaching 
methods in online learning during the lockdown period in COVID. The subject under study 
was STEM education using robotics. This research was designed to investigate whether playful 
learning techniques supported with technology could substantially increase engagement of 
learners. Thus, to aid the research, the following questions were formulated. 

Research Question 1: Incorporating playful learning in online STEM education 


through game design elements impacts learner’s engagement? 
Research Question 2: Are learning outcomes achieved through online STEM? 


4.4.2. Variable Selection 

As the integrated design framework mentions different types of learner engagements. 
To answer our first research question, we selected engagement as the variable to be tested, 
further subdivided into affective, physical (behaviour) and cognitive categories. 

For our second research question, student score attained in post-test was our selected 
variable. 

4.4.3. Hypothesis Formulation 

To compare engagement levels across control and experimental groups, we would use 


exploratory data analysis which is an approach to analysing data sets to summarize their main 
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characteristics, often with visual methods. Since choosing a statistical method is not necessary 
to evaluate findings in this case; we have not specified a hypothesis to be tested, rather we aim 
to conduct our investigation through graphical representation of data and compare results. 
The following hypotheses are formulated to assess whether learning outcomes are 
achieved in online STEM education. 
Ho: Marks obtained in post-test are equally high for playful and _ traditional 


methodology. 


Ha: Marks obtained in post-test are not equally high for both playful and traditional 

methodology. 

4.4.4. Experimental Study 

Experimental research was conducted for this study which falls under the quantitative 
paradigm. The type chosen was Quasi-experimental study. Quasi-experimental or field 
research is conducted in a natural setting as opposed to a laboratory set up. The variables are 
isolated, controlled and manipulated to suit the needs of the research question (Cohen et al., 
2018). However, the researcher does not exercise complete control over the research setting 
(experimental conditions or extraneous variables) as in an experimental design. Quasi- 
experimental designs are frequently used in social science and healthcare research (Harris et 
al., 2006) when randomisation or the use of control groups is unfeasible. 

For this research, non-equivalent control group post-test only design was employed. A 
non-equivalent group design is a quasi-experiment used to assess the relative effects of 
treatments that have been assigned to groups of participants nonrandomly. This design can be 
used in situations where randomisation is unfeasible because of ethical or practical 
considerations or it is too late or impossible to collect pre-test data. In studies that use the 


design, the intervention is implemented for the experimental group, which is then compared to 
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the non-equivalent control group and outcome measurements are taken from both groups (Polit, 
D.F.,& Beck, 2015). 
4.4.4.1. Control and Experimental Groups 

The purpose of a control group is to determine what outcome will occur in case the 
intervention, had not taken place. The experimental group is part of the population sample 
undergoing research intervention. For this study, control group and experimental group differ 
in their applied pedagogy. The control group is taught using the traditional online teaching 
method selected for imparting education during COVID, whereas, the experimental group is 
taught online using playful pedagogy and concept of robotics to make learning engaging for 
students. Both the groups are taught the same topic of “speed, distance and time” based upon 


STEM by the same instructor to keep the study as fair as possible. 


4.4.4.2. Post-test 
To collect data from the control and experimental groups, post-tests are conducted. To 
assess engagement levels during the study, an engagement survey was taken from the students 
whereas to assess learning, students from both the groups were asked to fill an MCQ based 
test. The post-test results of both control and experimental groups show the changes (if any) 


from the intervention. 


4.4.5. Content Design 
The topic chosen for the study was “Speed, Distance, and Time” in STEM education. 
Lesson was designed for control and experimental groups along the lines mentioned in the table 


below. 


Table 4.2: Content Design 


Groups Experimental Control 
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Content Delivery 


Playful approach using 


Elements of game design 
Robotics website 


Traditional content delivery by 
- Usual expression 
(Slides) 
- Worksheet 


Science: speed, time, distance 
Engineering: problem 
solving, coding 

Technology: robotics, online 
learning tool 


¢ Science: Speed, distance, 
time (concepts of 
physics) 

¢ Math: variable 
relationship (equation) 


STEM in the * Math: degrees, units, % * Technology: online 
lesson * Robotics learning platform 
* Coding ¢ Engineering: problem 
solving 
¢ MISSING: 
- Robotics 
- Coding 
* Recall concepts of speed, * Recall concepts of speed, 
distance, and time distance, and time 
* Calculate distance, speed and * Calculate distance, 
time speed, and time 
¢ Understand variable * Understand variable 
Learning relationships relationships 
Objectives ¢ Apply STEM concepts in real ¢ Apply STEM concepts 
life in real life 
¢ Understand programming 
instructions 
¢ Analyse angles and degrees 
¢ Plan and predict program 
execution 
* Create programs by using 
different sets of instructions 
4.4.5.1. Lesson Plan 


The lesson plan for control and experimental groups is outlined in the tables below. 


Table 4.3: Lesson Plan Experimental Group 


Timings Description 
00:00 — 00:05 Greetings 
Website Log in (https://vr.vex.com/) 
00:05 — 00:10 - Narrative Building 
* Code the Robot to reach the destination 
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- Competition (incentive) 
- Concepts 
* Coding Blocks 


00:10—00:15 | Activity 1: 
- Code Robot to reach on usual speed 
- Analyze speed time distance relationship 


00:15— 00:25 | Activity 2: 
- Code Robot to reach by changing its velocity 
- Analyze speed time distance relationship 


00:25 — 00:30 | Reflection and Wrap up 


Table 4.4: Lesson Plan Control Group 


Timings Description 


00:00 — 00:05 Greetings + Welcome Note 


00:05 — 00:15 * Objective 
* Brainstorming 
> Speed 
> Distance 
> Time 
¢ Variable relationship 


00:15 — 00:25 Word Problems 
Questions / Answers 


00:25 — 00:30 Homework and Wrap up 


To assess the content knowledge, following modes of assessment were included in 
content design. 
Assessment for learning (AFL) 
e Questions during lesson / activity to gauge learning 
Assessment of Learning (AOL) 
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e Post-test MCQ’s 


4.4.6. Content Delivery 

The research intervention was conducted online. Communication between teacher and 
students for both the groups was facilitated by Microsoft Teams as all the students already had 
their school login ID’s and were taking their usual online classes during COVID lock-down on 
Microsoft Teams. 

Topic “Speed, Distance, and Time” was delivered to both the groups such that STEM 
concepts are conveyed through the lesson and students understand real life applications of the 
topic being taught. Students were briefed learning objectives at the beginning of the lesson as 
follows. 

At the end of the lesson, students would be expected to: 

e Understand the correlation between speed, distance, and time 
e Analyse variable relationships 
e Derive further equations 
e Apply equations to solve problems 
4.4.6.1. Control Group 

The lecture was delivered to the control group through slides by using PowerPoint 

presentation on MS Teams which was the usual method of content delivery. 
Summary of the lesson is as follows: 
e Brainstorming: Which travels faster? Race car or fastest human being? 
e What is Speed and how do we measure it? 
e Why does a speedometer measure speed in km/h? 
e What is Distance? 


e What is the relationship of Speed and Distance with time? 
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e How do we calculate Speed, distance, and time? 
e Equations 
e Word Problems and real-life examples 
e Question/ Answers 
4.4.6.2. Experimental Group 
Content was delivered to the experimental group through inquiry-based instruction 
where students were immersed in discovery to achieve learning objectives of the lesson. 
Students were told that they would be playing to learn about “Speed, Distance, and Time”. 
As the integrated design framework incorporates elements of game design to achieve 
playful learning, online learning platform /ttps://vr.vex.com/ was used to engage students in 
play and robotics. Game design elements were embedded in the lesson as outlined in the table 


below. 


Table 4.5: Experimental Group Content Delivery 


Elements Description In Lesson 
Game Mechanics | the essential game play online robotics platform 
https://vr.vex.com/ 

Visual Aesthetic visual elements such as the online robotics platform 

Design overall look and feel of the https://vr.vex.com/ 
game and of the game 
characters 

Narrative Design storyline that is advanced via Narrative was built by the 
features such as cutscenes, in- facilitator. Students were given the 
game actions, dialogues and mission to program the robot to 
voice-overs crash the castle in the online 


learning platform. 


Incentive System | scores (points), stars, badges, Student who will complete the 
trophies, power-ups, and many __| missions in minimum time will 
other rewards win 

Content and Skills | the subject matter content and By performing the activity, learner 
skills that the game is designed | will discover the relation 
to teach e Speed a 1/ Time 


e Speed a Distance 


34 


Methodology 


Thus establishing, 
e Speed = Distance / Time 
e Distance = Speed x Time 
e Time = Distance / Speed 


Summary of the lesson is as follows: 

e Introduction to robotics through daily life examples such as smartphones, washing 
machines, microwaves, computers etc. 

e Programming robots means giving them set of instructions step by step for execution. 

e Learning objectives that students are expected to discover in lesson 

e Website log in and interface introduction 

e Narrative building 

e Continuous feedback and tasks to program robot 

e Review of speed, distance, and time concepts and variable relationships that students 
learnt while playing the game 

e Understand equations to calculate speed, distance, and time 

e Questions/ Answers 

4.4.7. Conducting Post-test 

Two post-tests were conducted from control and experimental groups to collect data 
and answer the research questions. The post-tests were created using Google Forms and links 
were shared with the students at the end of the lesson. 

To assess engagement levels, students were asked to fill out an engagement survey after 
the lesson. The survey was adapted from the engagement survey proposed by Chung et al., 
(2016) used to measure an individual’s behavioural, cognitive, and affective engagement. The 
Engagement survey by Chung et al. was written for use with 10 to 14-year-old respondents 
immediately after a science activity, whether in a class or in an informal learning context. It is 


available at http://activationlab.org/. According to Chung et al., (2016) “Our analysis of the 
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internal structure of the instrument indicates that valid inferences can be made regarding the 
overall engagement (i.e., a combination of affective, behavioural, and cognitive engagement) 
during an activity using responses across all items. Equally valid inferences can be made for 
two sub-factors of the scale, specifically an affective score or a behavioural/cognitive score. 
Responses to the cognitive and behavioural co-occur so tightly that separating those scores is 
not typically meaningful. Due to the self-report nature, the survey is not intended for high- 
stakes decisions about students (e.g., pass/fail determination, selection of program participants) 
or programs. Instead, the instrument is intended for formative feedback and/or for research 
purposes’. Hence, the survey fit well for our intended research’s comparative analysis between 
control and experimental group. It allows measuring different levels of engagement as outlined 
in the integrated design framework we are using in this research study. 

The second post-test, an MCQ based questionnaire aimed to test students’ knowledge. 
Students were required to solve the word problems by using equations to calculate speed, 
distance, and time and choose the correct option. 

4.4.8. Choosing Methods of Analysis 

To test a hypothesis, a statistical test needs to be conducted. The statistical test used 
should be determined by the data. 

There are 4 levels of data measurement: 

e Nominal: the data can only be categorized 

e Ordinal: the data can be categorized and ranked 

e Interval: the data can be categorized, ranked, and evenly spaced 

e Ratio: the data can be categorized, ranked, evenly spaced, and has a natural zero 

To prove our hypothesis, the data collected by post-test MCQ’s i.e. student scores will 
be analysed. In this case, the data is categorized, ranked and is evenly spaced so it falls under 


the category of interval. 
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Parametric or non-parametric tests are chosen depending upon whether the data is 
normally distributed or not. 


The Independent Samples ¢ Test is commonly used to test the following: 


e Statistical differences between the means of two groups 
e Statistical differences between the means of two interventions 


e Statistical differences between the means of two change scores 


Paired vs unpaired t-test 

A paired t-test is designed to compare the means of the same group or item under two 
separate scenarios and an unpaired t-test compares the means of two independent or unrelated 
groups. For this research study, we had two unrelated groups, experimental and control and we 


wished to find out statistical differences between the two interventions. 


Type of Data 


Independent 
Samples 


Non-parametric 


etric Dat 
Parametri a Date 


Mann-Whitney 
Test 


unpaired T-test 


Figure 4.1: Statistical Tests 


After running the normality test, the data was found to be non-normal. Therefore, non- 


parametric statistical test would be chosen. 
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The Mann-Whitney Test was chosen for this research as it is used to compare the means 


between two groups of non-parametric data. 


4.5. Summary 

A non-equivalent control group post-test only design was employed for this research. 
Students of Grade 6 and 7 from Army Public Schools were chosen for the control and 
experimental groups with 50 participants in each group (total sample size N=100). The topic 
“speed, distance and time’ was taught to both the groups by the same instructor using traditional 
and playful pedagogy for control and experimental groups respectively. 

This research was designed to investigate whether playful learning techniques impacted 
learner’s engagement and resulted in effective learning outcomes. 

To collect data from the control and experimental groups, post-tests were conducted 
after the lesson was delivered. To assess learner engagement, exploratory data analysis was 
used to visually represent data and draw comparisons. Whereas, hypothesis was statistically 
tested to check if pedagogy had any impact on student scores. The Mann-Whitney Test was 
chosen in this case as data was non-parametric. The results and findings of the study are 


explained in detail in the next chapter. 
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5. Data Analysis 


5.1. Overview 
Data was collected through post-tests after the implementation of the design experiment 
from control and experimental groups. Google Forms were used to collect the data which was 
then exported to excel sheets and tabulated. Statistical tests and exploratory analysis were 
conducted on the data using SPSS. Sample size was a total of 100 participants with 50 
participants in each control and experimental groups. 
5.2. Data Collected 
To answer our research questions, the data collected was divided in two parts and 
analysed. 
5.2.1. Part One: Engagement Survey 
To assess engagement levels, students were asked to fill out an engagement survey 
following the research intervention. The survey was adapted from the engagement survey 
proposed by Chung et al., (2016) used to measure an individual’s behavioural, cognitive, and 
affective engagement. Data was collected from the participants through Google Forms. The 
aim of this engagement survey was to answer the research question, “Incorporating playful 
learning in online STEM education through game design elements impacts learner’s 
engagement?” 
To analyse the data collected from the survey, we used Exploratory data analysis (EDA) 
which is a technique used by data scientists often employing data visualization methods to 


analyse and investigate data sets and summarize their main characteristics. 


Data was plotted visually to depict differences between engagement levels of control 
and experimental groups. Engagement levels were subdivided into Affective, Cognitive and 
Behaviour domains so to complement our integrated design framework deployed in the 


research study. 
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It was concluded that engagement levels of experimental group undergoing playful 
learning were higher as compared to the engagement levels of control group taught through 
traditional pedagogy. This pattern was observed similar across Affective, Cognitive and 


Behaviour domains. Refer to Figure 5.2 for visual representation of data. 


Bi Affective 
Wi Cognitive 
BW Behavior 


12.00 


10.00 


8.00 


6.00 


2.00 


00 


Control Experimental 


Group 


Figure 5.1: Data Analysis of Engagement Survey 
Furthermore, graphs were plotted to investigate engagement patterns across Class, 
Schools and Ages. It was concluded that experimental group showed higher levels of all 3 


engagement domains across all variables. Data is depicted in Figures below. 
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Figure 5.2: Engagement levels across Class 
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Figure 5.3: Engagement levels across Age 
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Figure 5.4: Engagement levels across Schools 


5.2.2. Part Two: Post-test Score 


To answer our research question “Are learning outcomes achieved through online 


STEM”, the following hypotheses were formulated. Tests were run on student ‘scores’ 


obtained in post-test MCQ’s. 


Ho: Marks obtained in post-test are equally high for playful and _ traditional 


methodology. 


H,: Marks obtained in post-test are not equally high for both playful and traditional 
methodology. 
5.2.2.1. Normality tests 


Normality tests were run on control and experimental group data comprising of 50 data 


values each; to check whether the data requires parametric or non-parametric statistical test to 


be run. The Shapiro Wilk test was used to test for normality in SPSS. 
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The tables below present the results from two well-known tests of normality, namely 
the Kolmogorov-Smirnov Test and the Shapiro-Wilk Test. The Shapiro-Wilk Test is more 
appropriate for small sample sizes (<= 50 samples) but can also handle sample sizes as large 
as 2000. If the Sig. value of the Shapiro-Wilk Test is greater than 0.05, the data is normal. If it 
is below 0.05, the data significantly deviates from a normal distribution. 


Hence, we can conclude from the tables below that “Score” is not normally distributed 


for control and experimental groups. 


Table 5.1: Normality Control Group 


Case Processing Summary 


Cases 
Valid Missing Total 
N Percent N Percent N Percent 
Score 50] 100.0% 0 0.0% 50] 100.0% 
Descriptive 
Std. 
Statistic Error 
Score | Mean 4.36 .106 
95% Confidence Lower 4.15 
Interval for Mean Bound 
Upper 4.57 
Bound 
5% Trimmed Mean 4.44 
Median 4.00 
Variance 562 
Std. Deviation .749 
Minimum 2 
Maximum 5 
Range 3 
Interquartile Range 1 
Skewness -1.316 337 
Kurtosis 2.166 .662 
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Tests of Normality 
Kolmogorov-Smirnov* Shapiro-Wilk 
Course | Statistic df Sig. Statistic df Sig. 
Score .283 50 .000 Boe 50 .000 


Table 5.2: Normality Experimental Group 


Case Processing Summary 
Cases 
Valid Missing Total 
N Percent N Percent N Percent 
Score 50| 100.0% 0 0.0% 50} 100.0% 
Descriptive 
Std. 
Statistic Error 
Score | Mean 3.60 185 
95% Confidence Lower 3.23 
Interval for Mean Bound 
Upper 3.97 
Bound 
5% Trimmed Mean 3.66 
Median 4.00 
Variance 1.714 
Std. Deviation 1.309 
Minimum 1 
Maximum 5 
Range 4 
Interquartile Range 3 
Skewness -.341 337 
Kurtosis -1.305 .662 
Tests of Normality 
Kolmogorov-Smirnov* Shapiro-Wilk 
Statistic df Sig. Statistic df Sig. 
Score 218 50 .000 847 50 .000 


Since the data was found to be non-normal, non-parametric statistical test was required 


for data analysis. 
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5.2.2.2... Mann-Whitney Statistical Test 
The Mann-Whitney U test is a non-parametric test that can be used in place of an 
unpaired t-test. It is used to test the null hypothesis that two samples come from the same 
population (i.e. have the same median) or, alternatively, whether observations in one sample 
tend to be larger than observations in the other the null and two-sided research hypotheses for 
the nonparametric test are stated as follows: 
e HO: The two populations are equal versus 
e H1: The two populations are not equal 
For data analysis of student score to assess learning outcomes, we chose Mann- 
Whitney non-parametric test. The test was run on the data (student scores attained in post-test 
MCQ’s) across different variables. Details are as follows. 
> Score across categories of Group 
The test was run on variable score across control and experimental groups to verify the 
following hypotheses. 
Ho: Marks obtained in post-test are equally high for control and experimental groups. 
Ha: Marks obtained in post-test are not equally high for control and experimental 
groups. 
The null hypothesis was rejected as result of statistical test. It was concluded that Marks 
obtained in post-test were not equally high for both control and experimental groups exhibiting 
traditional and playful pedagogy respectively. It was depicted that Control Group showed 


higher marks in score. Output of the test are listed in tables below. 


45 


Data Analysis 


Table 5.3: Mann-Whitney Test across Groups 


Hypothesis Test Summary 


Null Hypothesis Test Sig.” Decision 
The distribution of Score is the Independent-Samples .005 Reject the null 
same across categories of Group| Mann-Whitney U Test hypothesis 


Independent-Samples Mann-Whitney U Test 
Summary 
Total N 100 
Mann-Whitney U 870.000 
Wilcoxon W 2145.000 
Test Statistic 870.000 
Standard Error 136.807 
Standardized Test -2.778 
Statistic 
Asymptotic Sig.(2-sided .005 
test) 


Independent-Samples Mann-Whitney U Test 
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N=S0 
Mean Rank = 58.10 


Score 


Frequency 


Group 


Experimental Group 


N=50 
Mean Rank = 42.90 


2109S 


Frequency 


Figure 5.5: Mann-Whitney Test across Groups 
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> Score across categories of School 

The test was run on variable score across Army Public School Iqbal (Nowshera) and 
Army Public School Fort Road (Rawalpindi) to verify the following hypotheses. 

Ho: Marks obtained in post-test are equally high for Army Public School Rawalpindi 

and Army Public School Nowshera. 

Ha: Marks obtained in post-test are not equally high for Army Public School 

Rawalpindi and Army Public School Nowshera. 

The null hypothesis was retained as result of statistical test. It was concluded that Marks 
obtained in post-test were equally high for both schools that were part of the research. It was 
depicted that score was attained similarly for students irrespective of their schools. Participants 
of Rawalpindi were more as compared to Nowshera as APS Rawalpindi has more student 
strength. Output of the test are listed in tables below. 


Table 5.4: Mann-Whitney Test across Schools 


Hypothesis Test Summary 
Null Hypothesis Test Sig.” Decision 
The distribution of Score is the | Independent-Samples .637 Retain the null 
same across categories of Mann-Whitney U Test hypothesis 
School 


Independent-Samples Mann-Whitney U Test 
Summary 
Total N 100 
Mann-Whitney U 1126.500 
Wilcoxon W 1906.500 
Test Statistic 1126.500 
Standard Error 133.455 
Standardized Test -.472 
Statistic 
Asymptotic Sig.(2-sided ‘One 
test) 
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Independent-Samples Mann-Whitney U Test 


School 
Army Public School Fort 
Road Army Public School Iqbal 


N= 61 N=39 
Mean Rank = 51.53 Mean Rank = 48.88 
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Figure 5.6: Mann-Whitney Test across Schools 


> Score across categories of Age 
A popular nonparametric test to compare outcomes among more than two independent 
groups is the Kruskal Wallis test. The Kruskal Wallis test is used to compare medians among 
k comparison groups (k > 2) and is sometimes described as an ANOVA with the data replaced 
by their ranks. 
The Kruskal Wallis test was run on variable score across students aged 11-14 years to 


verify the following research hypotheses. Sample size was of 100 participants. 


Ho: Marks obtained in post-test are equally high for students between ages 11-14 years. 
H,: Marks obtained in post-test are not equally high for students between ages 11-14 


years. 
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The null hypothesis was retained as result of statistical test. It was concluded that Marks 
obtained in post-test were equally high for all participant students between ages 11-14 years. 
Output of the test are listed in tables below. 


Table 5.5: Kruskal-Wallis Test across Ages 


Hypothesis Test Summary 


Null Hypothesis Test Sig.” Decision 
The distribution of Score is the| Independent-Samples 456 Retain the null 
same across categories of Age Kruskal-Wallis Test hypothesis 


Independent-Samples Kruskal-Wallis Test 
Summary 
Total N 100 
Test Statistic 3.644a,b 
Degree of Freedom 4 
Asymptotic Sig.(2-sided 456 
test) 
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Independent-Samples Kruskal-Wallis Test 
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Figure 5.7: Kruskal-Wallis Test across Ages 
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> Score across categories of Class 

Mann Whitney test was run on variable score across students of Grade 6 and Grade 7 
to investigate differences in mean and verify the following hypotheses. 

Ho: Marks obtained in post-test are equally high for students in Grade 6 and Grade 7. 

Ha: Marks obtained in post-test are not equally high for students in Grade 6 and 

Grade 7. 

The null hypothesis was retained as result of statistical test. It was concluded that Marks 
obtained in post-test were equally high for students in both Grades. Strength of student 
participants in Grade 6 were more than in Grade 7. Output of the test are listed in tables below. 


Table 5.6: Mann-Whitney Test across Class 


Hypothesis Test Summary 


Null Hypothesis Test Sig. Decision 
The distribution of Score is the | Independent-Samples 741 Retain the null 
same across categories of Class} Mann-Whitney U Test hypothesis 


Independent-Samples Mann-Whitney U Test 
Summary 
Total N 100 
Mann-Whitney U 1287.000 
Wilcoxon W 2368.000 
Test Statistic 1287.000 
Standard Error 136.369 
Standardized Test .330 
Statistic 
Asymptotic Sig.(2- 741 
sided test) 
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Independent-Samples Mann-Whitney U Test 
Class 


Grade 6 Grade 7 


N=54 N= 46 
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Figure 5.8: Mann-Whitney Test across Class 


5.3. Summary 

Data collected from control and experimental groups through post-tests (MCQ’s and 
Engagement surveys) was analysed to answer the research questions. 

Higher engagement levels were depicted for experimental group as compared to control 
group. This data was analysed using graphical representation techniques. 

For post-test MCQ’s, hypotheses were formulated and tested using statistical methods. 
Null hypothesis was rejected in this case i.e. Marks obtained in post-test are not equally high 
for playful and traditional methodology. It was analysed that participants in control group 


attained higher score in MCQ’s compared to experimental group. 
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6. Discussion 


This research study was an attempt to compare playful pedagogy with traditional 
teaching methods in online learning during the lockdown period in COVID. The subject under 
study was STEM education using robotics. The content matter covered in the online lesson for 
both control and experimental groups was essentially the same. The difference was in the way 
the lesson was delivered. Robotics was part of the lesson for the experimental group only. The 
topic ‘speed, distance and time’ was taught to both the groups by the same instructor using 
traditional and playful pedagogy for control and experimental groups respectively. 

Students of Grade 6 and 7 from Army Public Schools were chosen for the control and 
experimental groups with 50 participants in each group (total sample size N=100) and a Quasi- 
experimental study was conducted. The pedagogy differed for both the groups such that the 
control group was taught through traditional online learning whereas experimental group was 
immersed in discovery learning through play. 

Designing a playful lesson plan takes more time than a traditional lesson plan as well 
as lesson delivery. The challenges and narrative building in playful learning demand more time 
and effort on behalf of the instructor for planning and execution. The lesson was mapped on 
30 minutes; however, lesson delivery took more time for experimental group as compared to 
control group. The students in experimental group were more engaged during the lesson in 
challenges and discovery of the online robotics platform. 

Introducing a new concept in class such as robotics requires the teacher to establish a 
connection with the students such that they develop familiarization with the concept instead of 
finding it foreign. This is important to maintain student engagement and interest during the 
lesson. For this study, robotics was introduced to the experimental group by relating robots 
with real life machines such as computers, televisions, automatic washing machines and 


microwaves etc. all these devices like robots require step-by-step instructions for execution of 
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commands. Building upon this foundation, it was easier for students to develop understanding 
of basic coding principles as they had to use block-based coding and program their robots in 
the online learning platform (vr.vex.com) to solve the challenges in their playful learning 
lesson. 

To ensure un-biasness in the STEM lesson delivery, real-life examples were quoted 
during the lesson to both groups to keep the lesson STEM focused and achieve desired learning 
objectives. 

As it is a challenge to keep the playful lesson structured, the teacher delivering the 
lesson focused on facilitating learners in play, offer feedback and guide learners to make 
discovery thereby ensuring that students do not de-track from lesson objective or lose 
engagement. In this case, it is also challenging to ensure that the student is learning through 
play and discovery and not being taught through transfer of knowledge from the teacher. For 
this reason, the teacher needs to be vigilant in offering timely feedback and optimum challenges 
which cater for diverse learner needs; as well as keep building the narrative upon requirement 


to keep the lesson playful and engaging. 


6.1. Findings of the Study 
The study was aimed to answer the following research questions by taking post-tests 
after implementation of research design from control and experimental groups to investigate 
results. 
> Research Question 1: Incorporating playful learning in online STEM education 
through game design elements impacts learner’s engagement? 
The experimental group showed a substantial increase based on_ graphical 
representation techniques and there was a significant difference between the post-test 
engagement survey means of the control and experimental groups. This indicates the benefits 


of using playful learning in teaching STEM online through robotics. 
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This answers our first research question. Hence, it is concluded that incorporating 
playful learning in online STEM education through game design elements impacts learner’s 
engagement positively across affective, cognitive and behaviour domains. 

> Research Question 2: Are learning outcomes achieved through online STEM? 

After the online research implementation, post-test was conducted to assess learning of 
control and experimental groups. This post-test was MCQ based and taken through Google 
Forms. To analyse data, following hypotheses were formulated and tested using statistical 
methods. 

Ho: Marks obtained in post-test are equally high for playful and _ traditional 

methodology. 

Ha: Marks obtained in post-test are not equally high for both playful and traditional 

methodology. 

It was analysed that the Null hypothesis was rejected for this study. There was a 
significant difference amongst the means of control and experimental groups across the 
variable score. It was investigated that participants in control group attained higher score in 
MCQ’s compared to experimental group. Hence, we can conclude that, marks obtained in post- 
test are not equally high for playful and traditional methodology. 

However, when the statistical tests were run across other variables to check whether 
score obtained had any relationship with school, age or the grade being taught, it was observed 
that these factors had not impacted marks of students and similar patterns of score were 
observed throughout all these variables. Difference in mean was only observed for control and 
experimental groups. Students being taught through traditional methodology scored higher 


marks in MCQ’s. 
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6.1.1. Relationship of Learning with Scores 

The research question for this study aimed to investigate whether learning outcomes 
were achieved by teaching STEM concepts online. In our hypotheses, we have related students’ 
obtained marks in post-test with learning. However, marks may not be the most accurate means 
of assessing learning. It is difficult to assess the extent to which course assessments reflect 
student learning because valid measures of learning are not available (Beleche et al., 2012). 

Testing can be reliable and valid, but it can measure only the inconsequential and 
distorted learning which is an unavoidable consequence of evaluation (Bryan & Clegg, 2006). 

Research states that assessment and testing should be set apart. Assessment is an 
informal gathering of information about students' knowledge through various ways of 
collecting information at various times and in different contexts. Testing, however, is formal 
and standardized and offers students scoring on the tasks they have performed. Testing is a 
single-occasion and timed exercise which is considered as the only way through which student 
learning can be measured. Many scholars do not agree that there is a single method of gathering 
data concerning student learning. Testing therefore is seen as just one component of the broader 
concept of assessment (Kulieke et al., 1990). 

Traditional assessment has emphasized on tests which show students' educational 
abilities. Research has pointed out the failure of traditional tests to capture the multi- 
dimensional aspects of what students have already learned (Mathies, 2000). According to 
learning scientists, subject matter content is easy to test but critical thinking and creativity is 
difficult to assess. Focus should be on long-term retention of knowledge and information and 
its transfer. Another method of assessment; Alternative assessment presents new ways of 
motivating and inspiring learners to explore and exploit dimensions of themselves as well as 
the world around them. Alternative assessment offers teachers a chance to realize their students' 


weaknesses and strengths in variant situations (Law & Eckes, 1995). 
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Similarly, authentic assessment involves students to carry out tasks which in one way 
or another involve them in some sort of problem-solving activities instead of testing rote 
learning and memorization. Authentic assessment puts forward a variant number of engaging 
tasks for the students in situations which are real world or simulation of real-world situations. 
As Wiggins (1990) claims, authentic assessment avails students with a wide range of skills and 
illuminates whether they have gained the ability to construct valid answers to the tasks 
presented. Moreover, he asserts that this kind of assessment sets a standardized criterion for 
scoring the tasks at hand by being highly reliable (Nasab, 2015). 

In our research, since we cannot claim post-test to be authentic form of assessment, we 
cannot assume marks to be a true reflection of learning. As STEM is an interdisciplinary hands- 
on curriculum that aims to inculcate 21“ century skills in learners, it cannot be tested by 
traditional assessment methods. The post-test for this research demanded students to answer 
multiple choice questions to assess their learning from the lesson being taught, this seems to 
be an inadequate method to assess learning and skills as it does not assess student performance. 

In a performance assessment, rather than choosing among pre-determined options, 
students must construct an answer, produce a product, or perform a hands-on activity. 
Performance assessments allow students to construct or perform an original response rather 
than just recognizing a potentially right answer out of a list provided. Hence, performance 
assessments can measure students’ cognitive thinking and reasoning skills as well as their 
ability to apply knowledge to solve realistic, meaningful problems (Darling-Hammond et al., 
2010). 

STEM education has received increasing attention in recent years. However, 
developing valid and reliable assessment of interdisciplinary learning in STEM is still a 


challenge. As STEM is interdisciplinary in nature, the connections across disciplines need to 
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be operationalized and properly assessed to provide targeted feedback to students. This 
demands further research and study. (Gao et al., 2020). 
6.2. Limitations 

The results of this quasi-experimental research study support the use of playful learning 
to teach STEM online by using robotics to increase student engagement. However, there are a 
few limitations to this study. 

Firstly, due to time constraints, only the topic “Speed, distance and time” was taught to 
both control and experimental groups (the former through traditional learning approach and the 
latter through playful learning by using an online robotics platform). A more comprehensive 
study conducted in greater detail should include more topics so that a stronger conclusion about 
the benefits of playful learning in STEM education can be made. 

Secondly, as the intervention was conducted during one lecture session (30 minutes), 
the implementation phase was very short. The lecture took more time to deliver for 
experimental group as compared to control group due to challenges and enhanced student 
engagement. However, the claim of playful learning advantages could be strengthened if the 
intervention was conducted for a longer period. 

The pedagogy for experimental group was playful whereas assessment techniques were 
traditional. This was a huge limitation to this study. Further research is required to develop 
authentic assessment techniques for a playful lesson. 

As the intervention was online due to COVID, technology was a must for both students 
and the teacher. High-speed internet and laptops were demanded specifically for students 
participating in experimental group as it was a requirement of the playful learning website. A 
limitation was that the website was not compatible with tablets or mobile phones whereas 
majority of student participants were using mobile phones to attend their online classes via MS 


Teams. List of students from APS Rawalpindi having desktops/ laptops at home was made by 
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conducting a pre-survey and only those students were made part of the experimental group. 
Students of APS Nowshera who were part of experimental group participated in the study by 
attending the class online from their school computer labs (when lockdown due to COVID 
eased) to minimize laptop/desktop requirement. For future studies, it is recommended that 
playful learning websites for education be developed for tablets and mobile phones so to bridge 
the current technology paradigm shift. 

Another limitation was that the students participating in the study from both control and 
experimental groups were familiar with technology and already attending online classes. Being 
students of a renowned education system, Army Public Schools, the students were computer 
literate and quick at grasping new concepts including block-based coding in short time. So, it 
is safe to assume that their technological skills were polished to say the least. A similar study 
should be conducted with participants with average and low technological skills to investigate 
their performance and learning using an online robotics platform. 

The integrated design framework has listed four types of engagement: affective, 
physical (behavioural), cognitive, and socio/cultural (Refer to Figure 3.1). For this study, we 
have not focused on role of socio/cultural engagement in playful learning. The post-tests have 
compared engagement levels of students of control and experimental groups in cognitive, 
affective, and behavioural domains but further study is needed to find out the factors which 
contribute to these multiple engagements resulting in playful learning as well as explore the 


role of socio/ cultural engagement. 
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7. Conclusion 


The aim of this research was to overcome the challenge of incorporating playful 
learning in an online learning environment and explore factors that make STEM e-learning fun, 
engaging and effective, keeping the students intrinsically motivated. This study was undertaken 
due to scarcity of research available in this field and the increasing demand of effective online 
learning owing to the education disruption created by the COVID-19 pandemic. 

The design framework implemented for this study is proposed by Jan L. Plass, Bruce 
D. Homer and Charles K. Kinzer. It describes the types of engagement created through game 
design elements to foster playful learning (Plass et al., 2014). These types include affective, 
cognitive, physical, and socio-cultural engagements. For this research, we have focused on 
studying the impact of affective, cognitive, and physical engagements of students created by 
the inclusion of game design elements in a playful online learning environment to teach a 
STEM topic. 

A quasi experimental study was conducted and non-equivalent control group post-test 
only design was employed for this research. Students of Grade 6 and 7 from Army Public 
Schools of Rawalpindi and Nowshera were chosen for the control and experimental groups 
with 50 participants in each group (total sample size N=100). The lesson was conducted online 
via MS Teams since students had already been using MS Teams to attend their online classes 
during COVID lockdown and were familiar with its procedures. The topic ‘speed, distance and 
time’ was taught to both the groups by the same instructor using traditional and playful 
pedagogy for control and experimental groups respectively. 

The control group was taught by their usual mode of lesson delivery. PowerPoint 
presentation was used to deliver the lesson to students in control group whereas, students in the 
experimental group were engaged in a playful learning experience by using an online robotics 


website (https://vr.vex.com). Students were given challenges to program their robots using 
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block-based coding. The website incorporated a Scratch like interface which facilitated ease of 
use for the students having no prior coding knowledge. 

This research was designed to investigate whether playful learning techniques impacted 
learner’s engagement and resulted in effective learning outcomes. The inclusion of game 
design elements; game mechanics, visual aesthetic design, narrative design, incentive system, 
content, and skills (Refer to Table 4.5) stimulated playful learning by using the robotics website 
to teach STEM concepts. Data was collected through post-tests after lesson delivery and 
analysed visually and statistically. 

Higher engagement levels were depicted for experimental group as compared to control 
group. This data was analysed using graphical representation techniques. Whereas, to assess 
whether learning had occurred during the intervention, hypotheses were formulated and tested 
using statistical methods. In this case, our null hypothesis was rejected. It was concluded that 
participants in control group attained higher score in post-test MCQ’s compared to 
experimental group. 

To conclude this study, employing playful techniques enhances engagement levels in 
students, however; the purpose of using such techniques should aid traditional learning instead 
of replacing it entirely. Furthermore, playful assessment techniques that aim to gauge 
knowledge instead of testing rote memory and bookish curriculum need to be devised. The best 
combination would be to embed playful learning in existing curricula for effective and 
engaging learning to occur. One way to do that is by employing elements of game design for 
learning using an online robotics platform as represented through this research study. 

Playful learning in online STEM still has a long way to go; needless to say, it helps 
create effective learning environments and benefits students. Learners stay intrinsically 
motivated by learning through online play that involves robotics. Coding principles help 


develop computational thinking in learners as well as build upon necessary 21* century skills. 
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It proves to be a fine alternative to classroom education in learning crises such as the recent 
pandemic hit. This research paves way for future studies in online STEM domain to explore 


and develop effective solutions that aid learning. 
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Appendix A 


Post-test MCQ’s 


The post-test questionnaire to assess learning outcomes is given below. 


Sr. no 


Question 


Correct Answer 


Marks 


1. 


After a meal, an earthworm moves at a 
distance of 45 cm in 90 seconds. Find the 
speed of the earthworm. 

a) 2cm/s 

b) 0.5 cm/s 

c) 4050 cm/s 


0.5 cm/s 


A car travels at 88 km/hr over a distance 
of 22 km. Find the time taken for the car 
to travel this distance. 

a) 0.25 hours 

b) 1 hour 

c) 3 hours 


0.25 hours 


A horse runs for 2 hours at a speed of 8 
miles per hour. How far does it run? 

a) 16 miles 

b) 15 miles 

c) 16kms 


16 miles 


A car travelling at a steady speed takes 4 
hours to travel 244 km. Find the speed of 
the car. 

a) 976 km/h 

b) 61 km/h 

c) 2km/h 


61 km/h 


A killer shark, attacking a fishing boat, 
swims at a speed of 13 m/s for half a 
minute. How far does it swim in this 
time? (Hint: half a minute = 30 sec) 

a) 390m 

b) 6.5m 

c) 0.43 m 


390 m 


Appendices 


Appendix B 


Post-test Engagement Survey 


The post-test survey to assess engagement levels of students is given below. 


Sr. No Prompt Sub-factor Response Options 
and Coding 
1. During this activity: I felt excited. Affect Very Much = 4 
Yes = 3 
Somewhat = 2 
No=1 
De During this activity: I felt happy. Affect Very Much = 4 
Yes = 3 
Somewhat = 2 
No=1 
3. During this activity: I felt bored. Affect Very Much = 1 
Yes=2 
Somewhat = 3 
No=4 
4. During this activity: I was daydreaming Cognitive Very Much = 1 
a lot. Yes =2 
Somewhat = 3 
No=4 
5. During this activity: I was focused on Cognitive Very Much = 4 
the things we were learning most of the Yes =3 
time. Somewhat = 2 
No=1 
6. During this activity: Time went by Behaviour | Very Much = 4 
quickly. Yes =3 
Somewhat = 2 
No=1 
he During this activity: I was busy doing Behaviour | Very Much = 1 
other tasks. Yes =2 
Somewhat = 3 
No=4 
8. During this activity: I talked to others Behaviour | Very Much = 1 
about stuff not related to what we were Yes =2 
learning. Somewhat = 3 
No=4 


